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A  TECHNIQUE  FOR  IMPROVING  DETECTION  AND  ESTIMATION  OF  SIGNALS 
CONTAMINATED  BY  UNDER  ICE  NOISE 


INTRODUCTION 

The  recent  analyses  of  FRAM  II  data  have  shown  that  strong  non-Gausslan 
noise  exists  in  the  frequency  domain.  This  technical  document  presents  the 
results  of  processing  FRAM  II  data  in  the  frequency  domain  using  data 
partitioning.  We  also  present  the  frequency  domain  envelope  distribution  at 
three  frequencies.  These  results  suggest  that  data  modeling  should  also  be 
considered  as  a  signal  processing  technique.  However,  here  we  shall 
concentrate  on  the  data  partitioning  results  and  leave  to  a  later  time  the 
discussion  of  the  results  of  data  modeling. 


MATHEMATICAL  PRELIMINARIES 

Let  the  received  data  x(i,q)  ■  x[(i+(q-l)M)h],  i»0,l,2,... ,M-1, 
q=l,2,...,n  where  h  is  the  interval  between  successive  samples,  be  composed  of 
an  additive  mixture  of  signal,  i nterference,  and  Gaussian  noise  of  the  form 

x(i,q)  *  m(i,q)s(i,q,a^)  +  c(i ,q)I(i ,q,a2)  +N(i,q),  (1) 


where 

s(i,q,ai)  is  the  transmitted  or  radiated  signal, 

I ( i , q, a'2 )  is  a  narrowband  interference  (ice  noise), 

N(i,q)  is  independent  Gaussian  noise, 

m(i,q)  represents  fading  or  multipath  effects  of  the  signal, 

c(i,q)  represents  fading  or  multipath  effects  of  the  interference,  and 

*t  is  a  random  variable  taking  on  two  values,  at*0,  at*l  with 

probabilities  P(at*0)»l-Lt,  P(at=l)=Lt  respectively,  for  t»l,2. 

The  random  variables  ai,  a2  model  transient  or  frequency  modulation 
components  of  the  signal  and  interference,  respectively,  in  the  sense  that  Li 
and  L2  represent  the  probability  of  the  signal  or  Interference  being  in  a 
particular  frequency  location  measured  over  the  observation  Interval.  The 
parameters  aj_,  a2,  m,  and  c  render  the  data  non-Gausslan  in  the  frequency 
domain.  In  general  these  parameters  are  also  functions  of  frequency;  however, 
for  the  sake  of  notational  simplicity,  we  shall  not  explicitly  show  this 
dependence.  The  medium  can  also  cause  the  received  signal  to  be  frequency 
modulated;  we  will  assume  that  the  frequency  modulation  caused  by  the  medium  is 
i ncorporated  i n  the  parameters  ai  and  a2 . 

The  discrete  fourier  transform  (DFT)  is  defined  as 


X( q.Fp)  >Vh/M 


M-l 


z 


x(i  ,q)exp(-jFpi) 


«  Xr(q,Fp)-j  XJ( q,Fp) , 


(2) 
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where  j ^ypl,  Fp=2irfph,  and  fp=p/Mh  Hz.  The  components  Xr(q,Fp) 
and  Xl(q,Fp)  signify  real  and  imaginary  parts,  respectively.  Temporal 
weighting  may  also  be  included  but  is  not  treated  here. 

If  we  assume  that  the  components  of  equation  (2)  are  mutually  independent, 
the  corresponding  spectrum  estimate  is 

P(F  )  =•  L1E(m2(q) )  S$(Fp)  +  L£  E(c2(q) )  Sj(Fp)  +  RN,  (3) 

where  we  have  assumed  slow  fading  with  respect  to  the  length  of  the  DFT  so 
that  in(i,q)=m(q)  and  c(i,q)=c(q),  and  Ss(Fp)  is  the  spectrum  estimate  at 
the  p-th  frequency  of  the  signal,  Si(Fp)  corresponds  to  the  p-th  frequency 
spectrum  estimate  of  the  interference,  and  R(j  represents  the  independent 
Gaussian  noise  spectrum  estimate.  The  components  and  \-Z  were  assumed 
to  be  multiplicative  in  the  frequency  domain. 

The  problem  to  be  addressed  here  concerns  processing  the  data  in  the 
frequency  domain  (as  given  by  equation  (2))  in  the  presence  of  strong 
non-Gaussian  interference.  This  problem  arose  as  a  consequence  of  the  FRAM  II 
data  analysis  study,  although  it  was  addressed  on  a  theoretical  level  in 
references  1  and  2. 

In  this  paper  we  will  present  under  ice  ambient  noise  data  which  are 
dominated  by  non-Gaussian  interference  in  both  the  time  and  frequency 
domains.  Then  we  will  show  that  by  processing  the  data  using  partitioning 
techniques,  significant  performance  improvements  in  terms  of  signal-to-noise 
ratio  (SNR)  are  possible  in  the  under  ice  environment,  assuming  that  the 
signal  level  is  much  smaller  than  the  interference.  This  assumption  is  not 
always  met  in  practice.  Reference  3  discusses  a  frequency  domain  processing 
method  that  treats  the  interference-free  case. 

The  theory  of  data  partitioning  methods  was  introduced  by  Ching  and  Kurz. 
Here  we  will  present  only  the  FRAM  II  data  partitioning  results.  Anyone 
interested  in  the  theoretical  aspects  of  partitioning  should  consult 
references  4  and  5. 

Another  approach  to  optimum  processing  is  data  modeling.  The  frequency 
domain  envelope  distribution  of  FRAM  II  data  is  also  presented  for  three 
frequencies  at  three  different  resolutions. 

FRAM  II  DATA  RESULTS 

Since  the  FRAM  II  data  results  of  under  ice  ambient  noise  have  been 
reported  in  detail  in  reference  6,  we  will  only  summarize  the  results  needed 
in  this  technical  document. 

The  data  analyses  are  composed  of  time  and  frequency  domain  statistical 
measurements.  The  time  domain  data  were  filtered,  sampled,  and  grouped  into 
records  of  1024  samples  each.  The  mean,  variance,  skew,  and  kurtosis  were 
then  estimated  for  each  record.  Over  time  intervals  consisting  of  hundreds  of 
records,  the  cumulative  distribution  function  (CDF)  of  the  energy  (square  of 
the  data  samples)  was  estimated  and  was  shown,  for  the  most  part,  to  be 
non-Gaussian  but  with  non  stationary  behavior  over  successive  intervals.  The 
time  domain  data  were  then  transformed  Into  the  frequency  domain  via  a  1024 
point  fast  Fourier  transform  (FFT).  Frequency  domain  statistics  were  then 
compiled  for  each  frequency  cell  for  both  the  real  and  imaginary  parts. 
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Figure  1*  shows  the  statistical  moments  for  the  time  domain  data,  which 
were  filtered  through  a  2500  Hz  lowpass  filter  and  then  sampled  at  a  1000  Hz 
rate.  Therefore  each  record  represents  a  time  Interval  of  about  0.1  second, 
giving  an  overall  data  length  of  10  minutes.  Some  Important  observations 
about  these  data  are  the  variability  of  the  variance  over  time  and  the 
significant  deviation  from  the  Gaussian  assumption  based  on  the  skew  and 
kurtosis  estimates.  We  found,  ty  processing  the  data  In  bands,  that  the 
variability  in  the  variance  was  due  to  higher  frequency  (greater  than  750  Hz) 
components.  The  kurtosis  Is  especially  Important  because  It  Indicates 
deviations  from  the  Gaussian  distribution  fcy  values  greater  or  less  than  3. 

The  values  greater  than  3  pertain  (In  many  cases)  to  distributions  that  are 
more  peaked  than  the  Gaussian  distributions  whereas  values  less  than  3 
correspond  (in  many  cases)  to  distributions  that  are  less  peaked.4.  For 
example,  a  purely  sinusoidal  signal  with  uniformly  distributed  phase  has  a 
kurtosis.  of  1.5.  Hone  of  the  records  in  figure  1  has  a  kurtosis  value  of  1.5 
although  some  are  near  2.  The  additive  Gaussian  noise  is  most  likely 
contributing  to  this  result  since  it  Is  known  that  kurtosis  Is  SNR  dependent 
(reference  3) . 

Figure  2  compares  the  power  spectral  density  (PSD)  (top  curve),  real  skew 
(middle  curve),  and  the  real  kurtosis  for  the  under  ice  data  which  have  been 
processed  with  a  10  Hz  resolution  and  averaged  over  1000  consecutive  FFT's. 
This  gives  a  total  time  interval  of  1.7  minutes.  The  data  clearly  Indicate 
non-Gaussian  noise  based  on  the  frequency  domain  skew  and  kurtosis.  Over  a 
relatively  flat  portion  of  the  band  as  seen  in  the  PSD  we  estimated  the 
amplitude  CDF  using  the  1000  consecutive  FFT's  for  both  the  real  and  imaginary 
parts.  The  results  (figure  3)  show  significant  deviation  from  a  Gaussian 
distribution  (dashed  curve)  for  both  the  real  and  imaginary  parts.  Later  In 
our  discussion  we  will  compare  these  results  with  the  original  data  after  it 
has  been  processed  by  partitioning. 

Another  data  set  of  under  ice  noise  is  given  in  figures  4  and  5.  The  data 
were  first  filtered  ty  a  100  Hz  bandpass  filter  centered  at  350  Hz,  were 
sampled  at  2000  Hz,  and  were  processed  in  records  of  1024  samples,  each  giving 
an  interval  of  approximately  0.5  second.  The  important  observations  in  figure 
4  are  that  the  variability  in  the  variance  is  greatly  reduced  in  this  band  and 
that  many  records  deviate  from  a  Gaussian  distribution  based  on  the  kurtosis 
estimate.  The  frequency  domain  results  (figure  5)  show  that  many  frequency 
locations  also  significantly  deviate  from  a  Gaussian  distribution  based  on  the 
frequency  domain  kurtosis  estimate.  In  addition,  a  60  Hz  tonal  and  some  of 
its  harmonics  are  present  In  the  PSD  estimate  In  figure  5.  The  corresponding 
kurtosis  estimate  shows  values  significantly  less  than  3.  An  in-depth 
theoretical  discussion  explaining  the  significance  of  these  results  for  a 
signal  propagating  in  a  medium  with  fading  or  multipath  effects  is  given  in 
reference  3. 
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FRAM  II  PROCESSING  RESULTS 

We  shall  discuss  the  results  of  processing  signals  In  the  frequency  domain 
here,  optimum  techniques  for  processing  time  domain  signals  in  non-Gaussian 
noise  can  be  found  in  references  7  -  10.  These  frequency  domain  processing 


*  All  figures  are  located  at  the  end  of  the  report. 

*  There  may  be  pathological  distributions  where  this  Interpretation  is  not 
justified  (12).  However,  deviations  from  3  Dertain  to  non-Gaussian 
distributions. 
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techniques,  which  we  shall  discuss,  represent  a  new  methodology  for  extracting 
signals  embedded  in  narrowband  non-Gaussian  noise.  References  1  and  2 
develop,  on  a  theoretical  level,  optimum  partitioning  techniques  in  sufficient 
detail  that  we  can  concentrate  here  on  the  results  of  processing  FRAM  II 
data.  First,  the  results  of  processing  the  10  Hz  resolution  frequency  domain 
data  by  partitioning  the  real  and  imaginary  parts  separately  will  be  given. 

Then  we  shall  present  the  frequency  domain  envelope  distribution  results  in  2, 

6,  and  10  Hz  resolution  cells.  These  results  suggest  that  noise  modeling 
should  also  be  considered  for  optimum  signal  processing  in  the  frequency 
domai n. 

Figure  6  compares  the  amplitude  CDF, of  the  partitioned  data  with  that  of 
the  original  data  for  both  the  real  and  imaginary  parts.  The  circles 
correspond  to  the  partitioned  results.  The  interesting  feature  is  that 
partitioning  for  this  data  set  produces  nearly  a  Gaussian  distribution.  This  f 

result  can  be  explained  by  noting  that  partitioning  is  equivalent  to  a 
nonlinear  transformation  that  significantly  reduces  the  high  amplitude 
excursions  and  leaves  the  smaller  amplitudes  unchanged.  It  should  also  be 
pointed  out  that  this  result  of  a  Gaussian  output  was  predicted  in  reference 
11.  The  performance  improvement  can  be  significant,  as  shown  by  the  following 
simple  example.  Suppose  it  is  desired  to  set  the  threshold  so  that  the  false 
alarm  rate  per  FFT  is  .0001.  Then,  assuming  a  small  SNR,  the  partitioned 
detector  would  have  approximately  a  15  dB  processing  advantage  over  a 
conventional  (linear)  detector,  as  shown  in  figure  6. 

Another  way  of  comparing  performance  is  based  on  the  asymptotic  relative 
efficiency,  which  is  essentially  a  ratio  of  the  output  SNR  of  two  detectors  as 
the  signal  approaches  zero  and  the  Integration  time  approaches  infinity.  As 
shown  in  reference  9,  this  turns  cut  to  be  a  ratio  of  output  variances  for  the 
two  detectors  under  noise-only  conditions.  Figure  7  compares,  for  the  10  Hz 
resolution  case,  the  CDF  of  the  partitioned  and  original  data  for  the  real 
part  only.  The  circles  represent  the  partitioned  data.  Again  note  the  close 
approximation  to  a  Gaussian  distibution  after  partitioning.  We  calculated  the 
output  variance  for  both  data  sets  and  found  that  partitioning  improves 
performance  in  this  case  by  about  5  dB  in  the  sense  that  its  variance  is 
decreased  by  5  dB  over  the  variance  of  the  original  data. 

Another  approach  to  optimum  processing  of  the  frequency  domain  FRAM  II 
data  is  to  model  the  data.  Figure  8  shows  a  plot  of  the  envelope  distribution 
at  the  output  of  an  FFT  at  three  frequencies  and  for  three  different 
resolutions.  We  considered  FFT's  with  10,  6,  and  2  Hz  resolutions  with 
time-resolution  products  ( TRP )  of  1000,  1000,  and  750,  respectively.  The 
vertical  scale  represents  the  envelope  (normalized  by  its  standard  deviation) 
in  dB,  and  the  horizontal  axis  is  the  exceedance  probability.  In  order  to 
better  visualize  the  tail  behavior  of  the  envelope  distribution,  we  included  a 
smal  1  number  of  adjacent  bins  in  the  estimate.  These  estimates  followed  ' 

approximately  the  curves  shown  in  figure  8.  In  this  way  we  were  able  to 
extend  the  tail  region  and  observe  its  trend.  The  solid  line  in  the  figure 
represents  a  Rayleigh  distribution.  As  can  be  seen,  the  data  deviate  from  the  t 

Rayleigh  distribution  for  all  three  cases  considered.  This  suggests  the 
possibility  of  modeling  the  envelope  distribution  of  FRAM  II  data  in  the 
frequency  domain. 
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SUMMARY 

We  have  presented  the  FRAM  II  data  analysis  results  and  have  shown  that 
the  ambient  noise  was  highly  non-Gaussian  in  the  time  and  frequency  domains. 

The  frequency  domain  results  are  new,  but  we  have  confirmed  them  try  a 
theoretical  analysis  and  by  comparison  with  other  data.  For  the  small  signal 
case,  two  tecnniques  were  considered  for  improving  detection  and  estimation  of 
signals  contaminated  with  under  ice  noise. 

In  the  first  technique,  we  employed  an  adaptive  partitioning  method  which 
approximates  the  optimum  nonlinearity,  as  derived  by  an  expansion  of  the 
log  likelihood  ratio,  for  the  particular  noise  distribution  prevailing  at  the 
time  a  decision  is  to  be  made.  Partitioning  transformed  the  CDF  of  the  FRAM 
II  data  into  a  Gaussian  distribution  and  thereby  decreased  the  threshold 
needed  to  maintain  a  prescribed  false  alarm  rate.  A  particular  example  showed 
that,  for  a  false  alarm  rate  of  .0001,  a  15  dB  improvement  in  performance 
could  be  achieved.  We  also  calculated  the  variance  of  the  original  and 
partitioned  data  and  found  that  a  gain  of  5  dB  in  performance  could  be 
achieved  based  solely  on  reduction  of  the  variance.  This  improvement  applies 
to  both  detection  and  estimation  of  signals  in  the  under  ice  noise  environment. 

Another  approach  to  optimum  processing  was  considered.  We  plotted  the 
envelope  distribution,  at  the  output  of  the  FFT  for  three  frequencies  for 
three  different  resolutions.  All  three  cases  deviated  from  the  Rayleigh 
distribution  which  suggests  that  data  modeling  should  be  considered  as  a 
signal  processing  approach  in  the  frequency  domain. 
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Figure  1.  Time  Domain  Statistical  Moments  for  2500  Hz  Band 
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Figure  2.  Frequency  Domain  Statistical  Moments 
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Figure  5.  Frequency  Domain  Statistical  Moments  for  2  Hz  Resolution 
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Figure  6.  Frequency  Domain  Data  and  Partitioned  Data 


Frequency  Domain  Data  and  Partitioned  Data  With  Gain 


Frequency  Domain  Envelope  Distribution 
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